Coverage-based Evaluation of Parser Generalizability
نویسندگان
چکیده
We have carried out a series of coverage evaluations of diverse types of parsers using texts from several genres such as newspaper, religious, legal and biomedical texts. We compared the overall coverage of the evaluated parsers and analyzed the differences by text genre. The results indicate that the coverage typically drops several percentage points when parsers are faced with texts on genres other than newspapers.
منابع مشابه
Dependency-based Evaluation of Minipar
In this paper, we first present a dependency-based method for parser evaluation. We then use the method to evaluate a broad-coverage parser, called MINIPAR, with the SUSANNE corpus. The method allows us to evaluate not only the overall performance of the parser, but also its performance with respect to different grammatical relationships and phenomena. The evaluation results show that MINIPAR i...
متن کاملFeature Engineering in Persian Dependency Parser
Dependency parser is one of the most important fundamental tools in the natural language processing, which extracts structure of sentences and determines the relations between words based on the dependency grammar. The dependency parser is proper for free order languages, such as Persian. In this paper, data-driven dependency parser has been developed with the help of phrase-structure parser fo...
متن کاملThe Power of the TSNLP: Lessons from a Diagnostic Evaluation of a Broad-Coverage Parser
We show a diagnostic evaluation of DIPETT, a broad-coverage parser of English sentences. We consider the TSNLP suite as a diagnostic tool, and propose an alternative broader-coverage test suite of test sentences extracted from Quirk et al. We compare the diagnostic effectiveness of the two suites, and draw a few general conclusions. The evaluation results were used to make significant improveme...
متن کاملSemi-Automatic Evaluation of the Grammatical Coverage of Machine Translation Systems
In this paper we present a methodology for automating the evaluation of the grammatical coverage of machine translation (MT) systems. The methodology is based on the importance of unfolded grammatical structures, which represent the most basic syntactic pattern for a sentence in a given language. A database of unfolded grammatical structures is built to evaluate the parser of any NLP or MT syst...
متن کاملGrammar & Parser Evaluation in the XTAG Project
In this paper we discuss several methods used to evaluate the XTAG parser and English grammar. We consider the methods proposed in the literature for grammar and parser evaluation, and give some empirical reasons for electing to use certain methods over others. We propose a general framework for evaluation, which is then used to evaluate the English grammar and parser developed as part of the X...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2008